The present invention relates to a coordinate input apparatus and method and coordinate input pointing device which input coordinates using a pointing device and, more particularly, to a coordinate input apparatus which inputs 2D/3D position coordinates by operating a pointing device at a remote place almost toward the information display screen of a display device and consequently inputs a graphic pattern or character or a command for controlling the system to a terminal device connected to the display device.
Conventional remote coordinate input apparatuses of following types are available.
For example, an apparatus of first type has a plurality of light-emitting elements laid out in a predetermined form on the pointing device side. The user appropriately tilts the pointing device to input coordinates. For this purpose, a light-receiving element is prepared on the fixed unit (stationary unit) side. The light-receiving element detects that the relative intensity between light beams emitted to the stationary unit side from the plurality of light-emitting elements on the pointing device side changes as the pointing device tilts, thereby calculating coordinates or the amounts of changes in coordinates.
An apparatus of second type irradiates a desired position on a display screen with a laser beam from a pointing device generally called a pointer. The image of the laser beam is detected by a camera behind the screen or on the front side of the display screen, thereby inputting coordinates or the amounts of changes in coordinates.
Conventional application examples of coordinate input apparatuses are a coordinate input apparatus in a system or application software for processing 3D coordinates, such as 3D CAD, 3D layout tool, 3D operation game, or 3D motion measurement apparatus, an operation apparatus, and a multimedia apparatus, amusement apparatus, and educational apparatus which serve as human interfaces and are used to receive the action or sign of a user as input information. These apparatuses for processing 3D coordinates include the following types.
1) An apparatus which uses a sensor supported by an articulated mechanism and detects the 3D motion or position of an object whose coordinates are to be input on the basis of mechanical contact between the sensor and the object whose coordinates are to be input, or mechanical deformation or dynamic acceleration of the sensor.
2) An apparatus which input 3D coordinates by a combination of 2D input operations of, e.g., a joystick.
3) An apparatus which calculates 3D coordinates by arithmetically processing a stereo image obtained by a plurality of video cameras sensitive to visible light or infrared light.
4) An apparatus disclosed in Japanese Patent Laid-Open No. 2000-112644, which comprises a pointing device that has a light-emitting element and points a 3D position, and a light-receiving element formed from a plurality of sensors for receiving light emitted from the light-emitting element and measuring the 3D position where the light-emitting element is present, so that the 3D coordinates at which the pointing device is located are calculated on the basis of a measurement value obtained by the light-receiving element.
Generally, such coordinate input apparatuses using light have the following problems.
In the apparatus of first type which has a plurality of light-emitting elements only on the pointing device, since the relative intensity of light is used as position information, the accuracy of the resultant coordinates or coordinate change amounts is low.
The apparatus of second type is not always safe because a laser beam is used. Additionally, since image processing is used, load on the computer is large. This poses a problem of response speed. Furthermore, even the necessity of a camera makes the hardware bulky.
For the conventional coordinate input apparatuses for processing 3D coordinates, the method 1) requires a large-scale mechanism and high manufacturing cost and therefore cannot cope with application purposes in a wide range. In the method 2), the user cannot intuitively input 3D coordinates, and no real-time and continuous processing can be performed. In the method 3), a continuous still image or moving image has an enormous data amount, and the arithmetic amount in calculating 3D coordinates is large. This increases load on the computer-side CPU and therefore makes real-time processing impossible and delays other processing.
Generally, 3D input includes 3D translation coordinate input (translational motion input) and 3-axis rotation angle input (rotational motion input). No apparatuses satisfactorily achieve both the input modes. The method 4) still has room left unsolved in this point.
The present invention has been made in consideration of the above situation, and has as its object to provide a coordinate input apparatus and apparatus, coordinate input pointing device, storage medium, and computer program, which are relatively simple and safe to handle and can accurately obtain coordinate data at a very high response speed.
A first coordinate input apparatus of the present invention is a coordinate input apparatus which comprises a fixed unit fixed at a predetermined position and a pointing device operated by a user and inputs coordinate data representing coordinates by the pointing device, wherein
the fixed unit comprises
first reception means for receiving a first signal from the pointing device, and
first transmission means for transmitting a second signal to the pointing device, and
the pointing device comprises
second reception means for receiving the second signal from the first transmission means,
coordinate calculation means for calculating the coordinate data on the basis of the second signal received by the second reception means, and
second transmission means for transmitting at least the coordinate data calculated by the coordinate calculation means to the first reception means as the first signal.
According to an aspect of the first coordinate input apparatus of the present invention, the second transmission means transmits a signal containing switch information representing a start of operation and a supplementary information signal to the first reception means as the first signal.
According to an aspect of the first coordinate input apparatus of the present invention, each of the first and second transmission means comprises a light-emitting element for emitting light.
According to an aspect of the first coordinate input apparatus of the present invention, each of the first and second reception means comprises a light-receiving element for receiving light.
According to an aspect of the first coordinate input apparatus of the present invention, the second reception means comprises a single-pixel light-receiving element and a plurality of line sensors laid out in different directions.
According to an aspect of the first coordinate input apparatus of the present invention, each of the line sensors comprises a linear photoelectric conversion element having a circulative charge transfer path constituted by a plurality of cells, and
the line sensor is designed to transfer charges simultaneously from all pixels of an arbitrary linear photoelectric conversion element to a charge transfer path connected to the circulative charge transfer path as a branch and then transfer the charges to the circulative charge transfer path, or transfer the charges simultaneously from all pixels of an arbitrary linear photoelectric conversion element directly to the circulative charge transfer path.
According to an aspect of the first coordinate input apparatus of the present invention, the line sensor circulates the charges on the circulative charge transfer path in synchronism with ON/OFF of an electronic shutter function and additionally accumulates the charges from the photoelectric conversion element at the same line position in every circulation of the charges, and
a signal read section is connected to a cell midway along the circulative charge transfer path to externally output a voltage proportional to the charges passing through the cell or output a voltage proportional to a difference in passing charges between two adjacent cells.
According to an aspect of the first coordinate input apparatus of the present invention, the electronic shutter function is turned on once at timings when the first transmission means emits light and the first transmission means emits no light to transfer the charges simultaneously from all pixels of the linear photoelectric conversion element to corresponding cells of the circulative charge transfer path once at each timing, and
the line sensor is designed to externally perform control to circulate the charges while additionally accumulating the charges by turning on/off the electronic shutter function and control to circulate the charges while fixing the electronic shutter function in an OFF state to stop additionally accumulating the charges.
According to an aspect of the first coordinate input apparatus of the present invention, the number of times of additional accumulation of the charges of the line sensor is controlled by the external control function in accordance with a reception amount of the light emitted from the first transmission means.
According to an aspect of the first coordinate input apparatus of the present invention, the second transmission means synthesizes the coordinate data with a timing signal for synchronizing operation of the line sensor with that of the first transmission means and transmits the synthesized signal to the first reception means as the first signal.
According to an aspect of the first coordinate input apparatus of the present invention, the first transmission means blinks in synchronism with the electronic shutter function in accordance with a predetermined timing sequence generated on the basis of the timing signal.
A first coordinate input method of the present invention is a coordinate input method in which a fixed unit fixed at a predetermined position and a pointing device operated by a user are used, and the pointing device is used to input coordinate data representing coordinates, comprising:
the first transmission step of transmitting a first signal representing a start of operation from the pointing device to the fixed unit;
the first reception step of receiving the first signal by the fixed unit;
the second transmission step of transmitting a second signal representing a direction of presence of the fixed unit itself to the pointing device;
the second reception step of receiving the second signal by the pointing device; and
the calculation step of calculating the coordinate data in the pointing device on the basis of the second signal.
According to an aspect of the first coordinate input method of the present invention, each of the first and second signals is an optical signal emitted/received between the fixed unit and the pointing device.
A coordinate input pointing device of the present invention is a coordinate input pointing device operated by a user and used to input coordinate data representing coordinates to a fixed unit fixed at a predetermined position, comprising:
transmission means for transmitting a first signal to the fixed unit;
reception means for receiving a second signal from the fixed unit; and
coordinate calculation means for calculating the coordinate data on the basis of the second signal received by the reception means,
wherein the coordinate data calculated on the basis of the second signal is transmitted to the fixed unit as the first signal.
According to an aspect of the coordinate input pointing device of the present invention, a signal containing switch information representing a start of operation and a supplementary information signal are transmitted to the fixed unit as the first signal.
According to an aspect of the coordinate input pointing device of the present invention, the transmission means comprises a light-emitting element for emitting light.
According to an aspect of the coordinate input pointing device of the present invention, the reception means comprises a light-receiving element for receiving light.
According to an aspect of the coordinate input pointing device of the present invention, the reception means comprises a single-pixel light-receiving element and a plurality of line sensors laid out in different directions.
According to an aspect of the coordinate input pointing device of the present invention, each of the line sensors comprises a linear photoelectric conversion element having a circulative charge transfer path constituted by a plurality of cells, and
the line sensor is designed to transfer charges simultaneously from all pixels of an arbitrary linear photoelectric conversion element to a charge transfer path connected to the circulative charge transfer path as a branch and then transfer the charges to the circulative charge transfer path, or transfer the charges simultaneously from all pixels of an arbitrary linear photoelectric conversion element directly to the circulative charge transfer path.
According to an aspect of the coordinate input pointing device of the present invention, the line sensor circulates the charges on the circulative charge transfer path in synchronism with ON/OFF of an electronic shutter function and additionally accumulates the charges from the photoelectric conversion element at the same line position in every circulation of the charges, and
a signal read section is connected to a cell midway along the circulative charge transfer path to externally output a voltage proportional to the charges passing through the cell or output a voltage proportional to a difference in passing charges between two adjacent cells.
According to an aspect of the coordinate input pointing device of the present invention, the electronic shutter function is turned on once at timings when the first transmission means emits light and the first transmission means emits no light to transfer the charges simultaneously from all pixels of the linear photoelectric conversion element to corresponding cells of the circulative charge transfer path once at each timing, and
the line sensor is designed to externally perform control to circulate the charges while additionally accumulating the charges by turning on/off the electronic shutter function and control to circulate the charges while fixing the electronic shutter function in an OFF state to stop additionally accumulating the charges.
According to an aspect of the coordinate input pointing device of the present invention, the number of times of additional accumulation of the charges of the line sensor is controlled by the external control function in accordance with a reception amount of the light emitted from the fixed unit.
According to an aspect of the coordinate input pointing device of the present invention, the transmission means synthesizes the coordinate data with a timing signal for synchronizing operation of the line sensor with that of the fixed unit and transmits the synthesized signal to the fixed unit as the first signal.
According to an aspect of the coordinate input pointing device of the present invention, the fixed unit blinks in synchronism with the electronic shutter function in accordance with a predetermined timing sequence generated on the basis of the timing signal.
A first storage medium of the present invention is a storage medium which stores a program for causing a computer to function as each component of the first coordinate input apparatus.
A first storage medium of the present invention is a storage medium which stores a program for executing a processing procedure of the first coordinate input method.
A first storage medium of the present invention is a storage medium which stores a program for causing a computer to function as each component of the coordinate input pointing device.
A second coordinate input apparatus of the present invention is a coordinate input apparatus which comprises a pointing device and a main body unit and inputs coordinate information indicated by the pointing device, wherein
the main body unit comprises first
detection means for detecting a first incident direction defined as an incident direction of light emitted from the pointing device with reference to the main body unit, and
first calculation means for calculating 3D translation information of the pointing device with respect to the main body unit on the basis of a detection result from the first detection means,
the pointing device comprises
second detection means for detecting a second incident direction defined as a relative incident direction of light emitted from the main body unit with reference to the pointing device, and
second calculation means for calculating 3-axis rotation information of the pointing device on the basis of a detection result from the second detection means, and
calculation results calculated by the first and second calculation means are substantially simultaneously output.
According to an aspect of the second coordinate input apparatus of the present invention, the first detection means has first line sensors in X and Y directions, the first line sensors detecting the first incident direction,
the main body unit further comprises a plurality of first light-emitting elements, and a single-pixel light-receiving element for receiving time series information from the pointing device,
the first line sensors comprise a plurality of line sensors in at least one of the X and Y directions, which are spaced apart by a predetermined interval, and
the plurality of first light-emitting elements comprise at least three light-emitting elements linearly laid out and spaced apart by a predetermined interval
According to an aspect of the second coordinate input apparatus of the present invention, the second detection means has second line sensors in X and Y directions, the second line sensors detecting the second incident direction, and
the pointing device further comprises at least one second light-emitting element, and a plurality of switches.
According to an aspect of the second coordinate input apparatus of the present invention, the first calculation means further calculates, on the basis of the first incident direction, 3D position coordinates of the pointing device with reference to a predetermined position, an amount of change in 3D position coordinates, and a moving amount corresponding to the 3D position coordinates.
According to an aspect of the second coordinate input apparatus of the present invention, the second calculation means further calculates, on the basis of the second incident direction, a tilt of the pointing device with reference to a predetermined direction, and an amount of change in tilt, and
the second light-emitting element transmits to the main body unit information obtained by synthesizing a calculation result from the second calculation means and switch information of the plurality of switches.
According to an aspect of the second coordinate input apparatus of the present invention, a Z-axis of the pointing device is defined as a direction indicating a front side of the pointing device or a direction from the pointing device to the main body unit,
X0- and Y0-axes of the pointing device are defined as coordinate axes that rotate along the Z-axis as the pointing device itself rotates along the Z-axis, and are perpendicular to each other and perpendicular to the Z-axis of the pointing device, and
the tilt is detected as a synthesis of xcex8x, xcex8y, and xcex8z defined on the basis of rotation along the three, X0-, Y0-, and Z-axes.
According to an aspect of the second coordinate input apparatus of the present invention, a Z-axis of the pointing device is defined as a direction indicating a front side of the pointing device or a direction from the pointing device to the main-body unit,
X0_fix and Y0_fix axes of the pointing device are defined as coordinate axes that do not rotate along the Z-axis as the pointing device itself rotates along the Z-axis, and are perpendicular to each other and perpendicular to the Z-axis of the pointing device, and
the tilt is detected as a synthesis of xcex8x fix, xcex8v_fix, and xcex8z defined on the basis of rotation along the three, X0_fix, Y0_fix, and Z-axes.
According to an aspect of the second coordinate input apparatus of the present invention, each of the first and second line sensors is a ring CCD formed from a linear photoelectric conversion element and having a circulative charge transfer path constructed by a plurality of cells,
charges are transferred simultaneously from all pixels of an arbitrary linear photoelectric conversion element to a charge transfer path connected to the circulative charge transfer path as a branch and then transferred to the circulative charge transfer path, or transferred simultaneously from all pixels of an arbitrary linear photoelectric conversion element directly to the circulative charge transfer path, and
the charges on the circulative charge transfer path are circulated in synchronism with ON/OFF of an electronic shutter function and additionally accumulated from the photoelectric conversion element at the same line position in every circulation of the charges.
According to an aspect of the second coordinate input apparatus of the present invention, in the line sensor, a signal read section is connected to a cell midway along the circulative charge transfer path to externally output a voltage proportional to the charges passing through the cell.
According to an aspect of the second coordinate input apparatus of the present invention, the electronic shutter function is turned on once during each of ON and OFF periods of light-emission-side blinking of incident light, and
the charges accumulated at each timing are transferred to adjacent cells on the circulative charge transfer path.
According to an aspect of the second coordinate input apparatus of the present invention, the signal read section can nondestructively read one of a voltage proportional to a difference of charges in an odd-numbered cell with respect to those of an even-numbered cell and a voltage proportional to a difference of charges in the even-numbered cell with respect to those of the odd-numbered cell.
According to an aspect of the second coordinate input apparatus of the present invention, the line sensor is designed to externally perform control to circulate the charges while additionally accumulating the charges by turning on/off the electronic shutter function and control to circulate the charges while fixing the electronic shutter function in an OFF state to stop additionally accumulating the charges.
According to an aspect of the second coordinate input apparatus of the present invention, the line sensor adaptively controls the number of times of accumulation in accordance with a light reception amount or an amount of accumulated charges.
According to an aspect of the second coordinate input apparatus of the present invention, blinking of light emitted from the pointing device is synchronized with an electronic shutter function of the first line sensors of the main body unit, and blinking of light emitted from the main body unit is synchronized with an electronic shutter function of the line sensors of the pointing device.
According to an aspect of the second coordinate input apparatus of the present invention, the second light-emitting element has a function of emitting light in accordance with a signal modulated by a carrier having a frequency much higher than a blinking frequency within a blinking period of light-emission-side blinking of the incident light, and transmits the time series information of blinking and the switch information on the pointing device by controlling presence/absence of modulation.
According to an aspect of the second coordinate input apparatus of the present invention, the main body unit comprises a bandpass filter having a center frequency substantially equal to a modulation frequency in the second light-emitting element, and a detection circuit, and detects the time series information and switch information transmitted by the second light-emitting element by passing a signal obtained by the first light-receiving element through the bandpass filter and detection circuit.
According to an aspect of the second coordinate input apparatus of the present invention, first light-emitting operation of the light emitted from the main body unit has a first correspondence relationship synchronized with second detection operation of the second detection means,
second light-emitting operation of the light emitted from the pointing device has a second correspondence relationship synchronized with first detection operation of the first detection means, and
the first and second correspondence relationships satisfy a predetermined condition.
According to an aspect of the second coordinate input apparatus of the present invention, for the synchronization, with reference to time of a leading edge or trailing edge of arbitrary lighting of-the blinking obtained from a time series signal detected by detection means which is characterized by having a function of emitting light in accordance with the signal modulated by the carrier having the frequency sufficiently higher than the blinking frequency within the blinking period of light-emission-side blinking of the incident light, and transmitting the time series information of blinking and the switch information on the pointing device by controlling the presence/absence of modulation, a timing sequence as a control signal of the ring CCD is started, and a series of operations are repeated every time coordinates of one point are processed.
A second coordinate input method of the present invention is a coordinate input method of using a pointing device and a main body unit and inputting coordinate information indicated by the pointing device, comprising:
the first detection step of detecting, by the main body unit, a first incident direction defined as an incident direction of light emitted from the pointing device with reference to the main body unit;
the first calculation step of calculating 3D translation information of the pointing device with respect to the main body unit on the basis of a detection result in the first detection step;
the second detection step of detecting, by the pointing device, a second incident direction defined as a relative incident direction of light emitted from the main body unit with reference to the pointing device; and
the second calculation step of calculating 3-axis rotation information of the pointing device on the basis of a detection result in the second detection step,
wherein calculation results calculated in the first and second calculation steps are substantially simultaneously output.
A second storage medium of the present invention is a storage medium which stores a program for causing a computer to function as each component of the second coordinate input apparatus.
A second storage medium of the present invention is a storage medium which stores a program for executing a processing procedure of the second coordinate input method.
A third coordinate input apparatus of the present invention is a coordinate input apparatus which comprises a pointing device and a main body unit and inputs coordinate information indicated by the pointing device, wherein
the main body unit comprises
first light-emitting means for emitting light, and
first detection means for detecting light emitted from the pointing device, and
the pointing device comprises
second light-emitting means for emitting light, and
second detection means for detecting the light emitted from the main body unit,
first light-emitting operation of the first light-emitting means has a first correspondence relationship synchronized with second detection operation of the second detection means,
second light-emitting operation of the second light-emitting means has a second correspondence relationship synchronized with first detection operation of the first detection means, and
the first and second correspondence relationships satisfy a predetermined condition.
According to an aspect of the third coordinate input apparatus of the present invention, the first detection means comprises a first line sensor having a first electronic shutter function synchronized with a first blink pattern period of the light which is emitted from the pointing device in accordance with a first blink pattern,
the first line sensor outputs a signal corresponding to a difference between charges accumulated during an ON period of the first blink pattern and those accumulated during an OFF period,
the second detection means comprises a second line sensor having a second electronic shutter function synchronized with a second blink pattern period of the light which is emitted from the main body unit in accordance with a second blink pattern, and
the second line sensor outputs a signal corresponding to a difference between charges accumulated during an ON period of the second blink pattern and those accumulated during an OFF period.
According to an aspect of the third coordinate input apparatus of the present invention, the second blink pattern period is twice the first blink pattern period,
blinking in the first blink pattern period is performed a total of two times during xc2xd the first blink pattern period in each second electronic shutter function ON period, and
the first ON period during the second blink pattern period and the ON period of the first blink pattern period are within the second electronic shutter function ON period, and
the first electronic shutter period of the first electronic shutter function is twice the second electronic shutter period of the second electronic shutter function,
the first electronic shutter function ON-period includes the twice ON periods of the second blink pattern period, and
an OFF timing of the first electronic shutter function matches one of OFF timings of the second electronic shutter function.
According to an aspect of the third coordinate input apparatus of the present invention, for a combination of the second electronic shutter period and the first blink pattern period and a combination of the first electronic shutter period and the second blink pattern period satisfy the predetermined condition in that
xe2x80x9cfor a first-order electronic shutter period Ts, a first-order blink pattern period of first-order blinking is 2*Ts, the number of times of ON operations during one first-order blink pattern period is one, the ON time of the first-order blinking is always included in an even-numbered (or odd-numbered) ON region of the first-order electronic shutter,
an nth-order blink pattern period of nth-order blinking is Ts*2n, the total number of times of ON operations is 2nxe2x88x921 times at the interval Ts during xc2xd the nth blink pattern period in each first-order electronic shutter ON period, for a phase relationship, a first ON operation in each blink pattern period is performed during the first-order electronic shutter ON period corresponding to the first-order blinking ON period,
the nth-order electronic shutter period is Ts*2nxe2x88x921, one nth-order electronic shutter ON period includes all ON periods during one nth-order blink pattern period, and the nth-order electronic shutter OFF period matches at least one of the first-order electronic shutter OFF periodsxe2x80x9d, and
the combination is included in combination of the nth-order blinking and electronic shutter and is a combination with different orders.
A third coordinate input method of the present invention is a coordinate input method of using a pointing device and a main body unit and inputting coordinate information indicated by the pointing device, comprising:
the first light-emitting step of emitting light from the main body unit;
the first detection step of detecting, by the main body unit, light emitted from the pointing device;
the second light-emitting step of emitting light from the pointing device; and
the second detection step of detecting, by the pointing device, the light emitted from the main body unit,
wherein first light-emitting operation in the first light-emitting step has a first correspondence relationship synchronized with second detection operation in the second detection step,
second light-emitting operation in the second light-emitting step has a second correspondence relationship synchronized with first detection operation in the first detection step, and
the first and second correspondence relationships satisfy a predetermined condition.
A third storage medium of the present invention is a storage medium which stores a program for causing a computer to function as each component of the third coordinate input apparatus.
A third storage medium of the present invention is a storage medium which stores a program for executing a processing procedure of the third coordinate input method.
A first computer program of the present invention is a computer program for causing a computer to function as each component of the first coordinate input apparatus.
A first computer program of the present invention is a computer program for causing a computer to function as each component of the first coordinate input pointing device.
A second computer program of the present invention is a computer program for causing a computer to function as each component of the second coordinate input apparatus.
A second computer program of the present invention is a computer program for executing a processing procedure of the second coordinate input method.
A third computer program of the present invention is a computer program for causing a computer to function as each component of the third coordinate input apparatus.
A third computer program of the present invention is a computer program for executing a processing procedure of the third coordinate input method.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof.